DOCUMENT RESUME 



ED 468 064 



UD 035 231 



AUTHOR 

TITLE 

INSTITUTION 
SPONS AGENCY 

REPORT NO 
PUB DATE 
NOTE 



AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Dee, Thomas S. 

Standards and Student Outcomes: Lessons from the "First Wave" 
of Education Reform. 

Harvard Univ., Cambridge, MA. Kennedy School of Government. 
National Inst, on Educational Governance, Finance, 
Policymaking, and Management ( ED/OERI ) , Washington, DC. 
PEPG/02-08 
2002-06-10 

35p.; Paper presented at "Taking Account of Accountability: 
Assessing Politics and Policy, " Kennedy School of Government, 
Harvard University, (Cambridge, MA June 10-11, 2002). 

Program on Education Policy and Governance, Taubman 306, 
Kennedy School of Government, 79 J.F.K. Street, Harvard 
University, Cambridge, MA 02138. Tel: 617-495-7976; Tel: 617- 
495-8312; Fax: 617-496-4428; e-mail: 
pepg_administrator@ksg . harvard . edu; Web site : 
http : / /www . ksg . harvard . edu/ pepg/ . 

Reports - Descriptive (141) — Speeches/Meeting Papers (150) 
EDRS Price MF01/PC02 Plus Postage. 

^Academic Achievement; * Academic Standards; Educational 
Attainment ; ^Educational Change; Elementary Secondary 
Education; Graduation Requirements; Labor Market; ^Outcomes 
of Education; State Standards 
Reform Efforts 



ABSTRACT 

This paper describes education reforms over the past 25 years, 
considering the consequences of earlier state-level standards. Most states 
introduced stricter, state-level standards for high school graduation in 
response to highly publicized concerns about student effort and public school 
quality. The key first wave of reforms consisted of a test-based performance 
standard, minimum competency testing (MCT ) , and a process standard, course 
graduation requirements (CGR) that mandated the amount of academic credits 
necessary in core academic areas. This paper describes these reforms and their 
implications for student outcomes, presenting new evidence on how they 
influenced such outcomes as educational attainment, labor market experiences, 
and high school curricula. Overall, the first wave of student-level standards 
have had positive and negative effects. For example, they led to reductions in 
educational attainments, particularly among black students. MCT led to 
reductions in curricular efforts. Higher CGR had pejorative effects on amounts 
of time spent doing homework and reading for pleasure. The standards also 
increased subsequent employment possibilities, and higher CGR were partly 
responsible for substantial academic upgrading of high school curricula. The 
paper recommends that ongoing public discussions about the desirability of 
centralized standards address how to value these reforms. (Contains 10 tables 
and 48 references.) (SM) 
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1. INTRODUCTION 

The "No Child Left Behind" Act, signed into law by President Bush in January, mandates the 
development of test-based student assessments and holding schools accountable for their academic 
performance relative to their state's minimum threshold (Robelon 2002). This bipartisan legislation will 
promote, and shape, the accountability systems that have been introduced in almost every state over the last 
several years. The accountability policies recently adopted at the state level have taken several forms 
including the publication of "report cards" and ratings for schools, teacher evaluations coupled with merit 
pay and the legal authority for states to control or close failing schools (Meyer et al. 2002). An increasing 
number of states also hold students directly accountable by withholding grade promotion or high school 
graduation for low performance on tests. The impetus for output-based accountability has grown out of the 
widely held perception that the long-standing focus of prior reforms on educational inputs and processes 
has been relatively unproductive. In particular, proponents of accountability policies argue that we do not 
have reliable information about how to systematically use educational programs and resources to improve 
student outcomes. According to this line of reasoning, the idiosyncratic nature of educational production 
implies that output-based incentives provide a more reliable method for enhancing productivity (e.g., 
Hanushek and Raymond 2001). However, as suggested by the title of the recent Federal legislation, 
another clear intent of accountability programs has been to close the performance gap between advantaged 
and disadvantaged students, in particular the gap between white and minority students. 

Critics of these policies suggest that explicit standards may actually exacerbate that performance 
gap, particularly in the absence of other systemic reforms related to local control, teacher training and 
available resources (e.g., Mumane and Levy 2001). Another major concern with standard-based reforms is 
that they may promote an undesirable narrowing of teaching styles and student curricula (Kohn 2001 , 
Mumane and Levy 2001). A recent national survey of public school teachers suggests that those concerns 
could be well-founded (Doherty 2001). While a majority of surveyed teachers (87 percent) support 
establishing higher standards, nearly 70 percent also thought that their teaching overemphasized testing to 
the detriment of learning in other important areas (Doherty 2001). A large majority of these teachers also 
opposed using state tests as the sole basis for grade promotion and graduation. Nonetheless, there have also 
been some early indications that the states making the strongest recent gains in measured student 
achievement have been those that aggressively implemented new standards and assessments (Olson 2001). 
However, the proper interpretation of the recent experiences within particular states has been the subject of 
considerable controversy (e.g., Schrag 2000, Greene 2000) and many of the new state policies have been 
implemented too recently to be evaluated with currently available data. 

The premise for this paper is that we may gain useful insights into these controversies by looking 
back to consider the consequences of earlier state-level standards. Over the last twenty-five years, almost 
every state introduced stricter, state-level standards for high school graduation in response to highly 
publicized concerns about student effort and the quality of public schools. The key "first wave" reforms 
consisted of a test-based performance standard, minimum competency testing (MCT), and a "process" 
standard, course graduation requirements (CGR) that mandated the amount of academic credits that must 
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be earned in core academic areas. Though the adoption of these reforms occurred with much fanfare, there 
has been surprisingly little study since then of their consequences. In this paper, I describe these reforms 
and what we currently know about their implications for student outcomes. I also present new evidence on 
how these reforms influenced a variety of student outcomes (e.g., educational attainment, labor market 
experiences, high school curricula) in empirical specifications that address the possible shortcomings of 
prior evaluations. I conclude by discussing what this broad set of results may contribute to our 
understanding of the currently evolving state and Federal experiments with standards-based reform. 

2. THE "FIRST WAVE" OF EDUCATION REFORM 

Much of the ongoing public interest in reforming public education can be traced to the mid- 1970's 
and the widely discussed perception that student test scores and the quality of public schooling were in 
decline. 1 In particular, critics of that period emphasized that a high school diploma, once a significant and 
hard-earned, personal accomplishment, had been debased through the abuses of social promotion and the 
tolerance of low academic standards (e.g., Popham 1981). Politicians at the state level proved highly 
responsive to these concerns and began enacting a variety of new standards and regulations now known as 
the "first wave" of education reform. 2 The earliest manifestation of these centralized reforms was the 
widespread adoption of a test-based performance standard: minimum competency testing (MCT). 
Beginning in 1975, nearly every state introduced new MCT programs designed to assess students' basic 
skills (Pipho 1978). Most of these programs were simply intended to identify low-performing students and 
to direct them to sources of remediation. However, several of these states also mandated that students pass 
an MCT in order to graduate with a standard diploma. By 1992, the graduating high school seniors in 
fifteen states were required to pass an MCT (see Table l). 3 Typically, students would first sit for these 
exams in the 9 th or 10 th grades and have multiple opportunities for re-tests. The conventional wisdom 
regarding these test-based diploma sanctions has been that they were "legislated as a lion but implemented 
as a lamb" (Catterall 1989). Specifically, the MCT standards typically required that students demonstrate 
basic math and reading skills at only an 8 th or 9 th grade level. Furthermore, in response to failure rates on 



1 Prior to this, the most extensive public discussions of education reform were occasioned by the Sputnik 
launch in 1957. Ironically, the discussions in the 1970s were motivated in part by the declines in Scholastic 
Aptitude Test (SAT) scores, a trend with ambiguous implications since an increasing proportion of students 
were likely to take the test (CBO 1986, 1987). 

2 These "top-down" reforms consisted of state-level standards and regulations that influenced teachers (e.g. 
licensing and salaries), schools (e.g., curricula, length of school day) as well as students (e.g. graduation 
requirements). In contrast, the subsequent, "second-wave" of reform stressed decentralized improvements 
like school-based management, teacher professionalism and school choice (Saban 1997). The second wave 
reforms are sometimes viewed as a response to the failure of the first wave (e.g., Chubb and Moe 1990). 

3 The enactment of this requirement was often delayed for several years in order to provide students with 
adequate notice, to allow the state to test and develop their assessments, to allow schools to adjust curricula 
and in response to court challenges (e.g., Debra v. Turlington in Florida). I identified the existence and 
effective date of MCT requirements for graduation by drawing on several sources including Jacobs (in 
press). Bishop and Mane (2001), publications from the Education Commission of the States and Lexis- 
Nexis searches of state newspaper accounts. 
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initial tests that were deemed politically unacceptable, these standards were sometimes lowered (Catterall 
1989). As a consequence, the ultimate pass rates among high school seniors were extremely high (Serow, 
1984). However, whether MCT has had a more substantial influence on dropout, rates is an open empirical 
question since the attrition of discouraged students may make the ultimate pass rates misleading. 
Furthermore, a full consideration of MCT policies should also consider their effects on other outcomes that 
are relevant to all students (e.g., student curricula and labor market experiences). 

The adoption of "first wave" reforms accelerated more dramatically in the early 1980's after the 
publication of several panel reports, which were highly critical of public education. The most widely 
discussed of these reports, A Nation at Risk (National Commission on Excellence in Education 1983), 
emphasized the need for higher expectations and standards for high school graduates. In particular, the 
report alleged that the combination of a "cafeteria-style curricula" and "extensive student choice" meant 
that too many students pursued a diffuse and unchallenging course of study. The report recommended that 
states respond with new high school graduation standards mandating a minimum amount of course taking 
in core academic areas. The report specifically recommended a "New Basics" curriculum requirement 
consisting of 4 years of English and 3 years each of social studies, science and mathematics. 4 Again, 
politicians proved highly responsive to the strong public interest in these policies. By 1992, nearly every 
state had increased its course graduation requirements (CGR) in the four core academic areas (see Table 
l). 5 However, in all but three states (Florida, Louisiana and Pennsylvania), the new CGR fell short of the 
"4/3/3/3" standard recommended by A Nation at Risk. 

3. STANDARDS AND STUDENT ACHIEVEMENT 

The fundamental motivation for these two first-wave reforms (MCT and CGR) was simply to 
promote student effort and learning, making high school diplomas "worth the paper they're written on." 

Are new educational standards likely to have beneficial effects on student achievement? Prior 
commentators on this issue disagree sharply for reasons that are worth considering explicitly. Proponents 
of higher standards (e.g., Betts and Costrell 2001) make the straightforward claim that the "incentive 
effects" of such policies will raise the level of achievement among those students who would pass under a 
weak standard and choose to increase their effort to meet the new standard. Betts and Costrell (2001) also 
suggest that those students whose prior levels of effort would clearly imply failing or passing both 
standards (i.e., those at the top and bottom ends of the ability distribution) will not have any incentive to 
change their behavior. However, they also recognize that the incentive effects for some students who 



4 The amount of credit for a course is typically defined in terms of "Carnegie units," which represent a 
standardized amount of time spent studying a subject over a full academic year. One unit represents 
successful completion of a class which averages fifty minutes per day, five days per week for 180 school 
days (Lillard and DeCicca 2001). A Nation at Risk also recommended 0.5 Carnegie units of computer- 
related courses and 2 of a foreign language for college-bound students. 

5 State-year data on CGR were drawn from several sources, most of which were published by the Education 
Commission of the States. 
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marginally passed under weaker standards will promote discouragement and reduced effort. They 
recommend targeted policies to attenuate these losses. 

However, the potential benefits of higher standards are not necessarily limited solely to those 
marginal students who choose to increase their effort. For example. Bishop (1999) discusses how a high, 
external standard can limit the "nerd harassment" and peer pressure that encourages high-ability students to 
shirk educational effort. It is also possible that standards can generate broader educational gains through 
general increases in educational expectations and school productivity. Furthermore, even those students 
who fully anticipate dropping out of high school may be compelled in the short term to greater educational 
effort through curricular mandates such as course graduation requirements (CGR). The "sorting effects" of 
higher standards may also lead to passive labor market rewards (Betts and Costrell 2001). Specifically, if 
educational attainment functions as a signal of unobserved individual ability, higher standards could 
increase the attractiveness to employers of all students by increasing the average level of ability among 
both dropouts and graduates. 

The critics of standards-based reform emphasize the many negative consequences associated with 
the expected reductions in educational attainment. Furthermore, they note that the reductions in 
educational attainment are disproportionately likely to be among those whose poor socioeconomic priors 
make it unusually difficult to meet new standards. In particular, several observers suggest that higher 
standards will exacerbate the troublesome performance gaps between black and white students (e.g., Serow 
1984, Mumane and Levy 2001, Philips and Chin 2001 ). 6 However, the critics of standards also suggest 
that these reforms will have other, pejorative effects that harm all students. For example, the introduction of 
high-stakes testing such as minimum competency tests (MCT) may lead to a narrowing of teaching styles 
and curricula (i.e., "teaching to the test") that comes at the expense of substantive learning (Mumane and 
Levy 2001). Furthermore, the establishment of minimum competency tests (MCT) and stricter course 
graduation requirements (CGR) may suggest to students that learning for its own sake is not worthwhile. 

In particular, these standards may encourage otherwise high-achieving students to avoid challenges and 
simply choose the path of least resistance to satisfying their requirements (Phillips and Chin 2001). 
Interestingly, the authors of A Nation at Risk made a similar allegation, suggesting that minimum 
competency tests (MCT) were inadequate since they would become "maximum" standards and lower 
expectations for high-ability students. 

PRIOR EMPIRICAL EVIDENCE 

The prior discussion suggests that there are fundamental, policy-relevant issues about the 
educational consequences of higher standards that can only be informed by empirical evidence. Given this, 
one might expect that the first-wave reforms have been subjected to exhaustive empirical evaluation. 

6 The consequences of higher standards for the racial gap in educational performance may actually be 
driven by more than simple differences in socioeconomic backgrounds. Higher standards and high-stakes 
testing may also harm minority students if they generate "stereotype threat"; academic underperformance 
due to the risk of confirming negative stereotypes (Steele 1997, 1998). 



Surprisingly, there is relatively little empirical evidence on the consequences of these policies that would 
allow us to sort through these conflicting theoretical predictions. Furthermore, what evidence is available 
is often directly contradictory. Most of the prior studies have focused on how minimum competency tests 
influenced student achievement as measured by test scores. For example, Frederiksen (1994) compared 
state-level NAEP scores before and after the introduction of minimum competency tests (i.e., 1978 and 
1986). He found that math scores rose in "high-stakes'* states (those that use MCT for grade promotion and 
graduation) relative to "low-stakes" states (those that have no stated use for their exams). Interestingly, 
these basic "difference-in-differences" comparisons also suggested that these achievement gains were 
concentrated among 9 and 13-year olds, not 17-year olds. However, both Bishop et al. (2001) and Jacob 
(in press) present evidence that residence in an MCT state is unassociated with either the average level or 
gain in student test scores. 7 

Most studies also find that residency in an MCT state has no apparent effect on the overall 
probability of completing high school (e.g., Catteral 1987, Lillard and DeCicca 2001, Bishop and Mane 
2001, Jacobs in press). However, there is evidence that, for low-achieving students in MCT states, the 
chances of graduating from high school are lower (Bishop and Mane 2001, Jacobs in press). Bishop and 
Mane (2001) also evaluate the effects of minimum competency testing on the probability of entering 
college and on early labor market experiences. They found, among high-achieving students, a positive 
association between residence in an MCT state and the likelihood of entering college. They also find that 
the students from MCT states have subsequently higher earnings, even after controlling for their 
educational attainment. 8 

The evidence on how course-taking standards influenced achievement and educational attainment 
is more limited. Some studies find that the adoption of higher CGR led students to pursue more demanding 
and academic curricula (e.g., Cluneand White 1992, U.S. Department of Education 1997). 9 However, 
studies attempting to identify the effects of new CGR on test scores have found no significant effects (e.g., 
Bishop et al. 2001). 10 The available evidence on how CGR influenced educational attainment is also 
mixed. Hoffer (1997) finds no effect on the probability of completing high school while Lillard and 
DeCicca (2001) report statistically significant, negative effects. Bishop and Mane (2001) also find negative 
associations between living in a state with CGR and the probabilities of completing high school and 
entering college. However, the increased propensity to drop out of high school appears to be offset by an 

7 However, Bishop (1998) and Bishop et al. (2001) presented evidence that curriculum-based exit exams 
do raise test scores. These evaluations relied on cross-national comparisons as well as other cross-sectional 
comparisons based on data from Canadian provinces and U.S. states. 

8 Furthermore, these gains appear among both high and low-achieving students. Similarly, Bishop and 
Mane (1999) found that attendance at an MCT high school was positively associated with wages and 
earnings. 

9 However, since these studies only considered the transcripts of graduated seniors or students with at least 
16 Carnegie units, this evidence may merely reflect the reform-driven attrition of students who took 
relatively few academic courses. 

10 Furthermore, studies that exploit the variation in graduation policies across schools also find no effects 
on test scores (e.g., Hoffer 1997). 
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increased probability of acquiring a general equivalency degree (GED). They also find that, conditional on 
educational attainment, exposure to higher CGR are negatively associated with subsequent wage rates. 

4. EVIDENCE FROM THE 1 990 PUMS 

Overall, what does this body of evidence suggest about the desirability of standards-based 
reforms? Bishop et al. (2001, page 312) argue that the comparative effects of CGR and MCT on 
educational attainment and labor market outcomes imply that MCT are a relatively attractive policy. 
However, these sorts of policy inferences may be premature for two reasons. First almost all of the 
available empirical evidence motivating this conclusion relies effectively on cross-sectional comparisons of 
students who reside in states with different policies. 11 Since the considerable variation in individual 
educational outcomes across states reflects a variety of cultural, socioeconomic and political determinants, 
this approach may generate substantive biases. In particular, the unobserved but state-specific determinants 
of educational achievement are also likely to be associated with each state's propensity to adopt high school 
graduation requirements like MCT and CGR. Relying on a cross-state identification strategy to estimate 
the effects of these policies could, therefore, produce biases of an uncertain direction. 12 A second 
drawback of prior empirical studies is that they have not directly addressed claims about whether these 
graduation standards would be particularly harmful or beneficial to minority students. In this section, I 
present new evidence that addresses both of these concerns by relying on individual-level data from the 
1990 Public Use Microdata Sample (PUMS). 

DATA AND SPECIFICATIONS 

The 1990 PUMS consists of approximately 12 million respondents (5 percent of the population) 
who completed the long form questionnaire to the decennial Census (U.S. Bureau of the Census 1993). 

One useful feature of the PUMS is that, because it contains individuals from multiple birth cohorts within 
each state, the resulting identification strategy can rely on the within-state variation in MCT and CGR 
instead of the cross-state variation. Furthermore, the large number of respondents in the PUMS implies 
increased statistical precision and, in particular, a better ability to detect race-specific responses to the new 
graduation standards. My extract from the PUMS data consists of the 1,348,766 white (non-Hispanic) and 
black respondents who were aged 18 between 1980 and 1988 and bom in one of forty nine states (Ruggles 



1 1 One exception is the difference-in-difference comparisons reported by Frederiksen (1994). Another is 
Lillard and DeCicca (2001, page 467, model 4) who find that higher CGR increase the probability of 
dropping out. However, they do not address the effects of MCT in these fixed effect models. 

12 For example, one plausible conjecture is that the states most willing to adopt higher standards are those 
with relatively high-achieving students. However, it is similarly plausible to suspect the opposite: that the 
states with an unobserved propensity for low student achievement would adopt higher standards more 
aggressively. 
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et al. 199 7). 13 Two of the outcome variables defined for each respondent identify educational attainment, a 
binary indicator for high school graduation (mean=0.858) and another for college entrance (mean=0.519). 14 
I limited the sample to those who were at least 18 by 1988 because of the biases that could be generated by 
state-specific trends in the "incomplete spells" of high school completion and college entrance among 
cohorts that were younger at the time of the Census interview (Angrist and Evans 1999). The other 
dependent variables reflect the labor experiences of each PUMS respondent One is a binary indicator for 
employment participation (mean=0.745), which is defined for all respondents. 15 The other is the natural 
log of average weekly wages, which is only defined for 1,143,352 respondents. This wage variable is the 
ratio of pre-tax wage and salary income reported for the previous calendar year and the corresponding 
number of weeks worked (mean=327). 

The basic specification used for regression models based on these data is: 

Yi s t — px ist + yZ st + ps + Vt + Sj s t 

where Y ist is the dependent variable and the matrix, X ist , includes observed, individual-level traits. In most 
models, these controls simply include binary indicators for race and gender. However, in the models for 
labor market outcomes, these controls also include measures of educational attainment (i.e., separate 
dummy variables for high school graduates, those with some college and those with bachelor degrees) and 
a dummy variable for whether the respondent attended school within the last year. 16 The terms, p s and v t , 
represent fixed effects specific to each state of birth and year of birth. The term, s ist , is a mean-zero random 

17 

error. 

The matrix, Z st , includes determinants that were specific to the birth cohorts within each state. 
These determinants include the two independent variables of interest: dummy variables that reflect the state 
MCT and CGR policies in effect for each birth cohort at age 18. One dummy variable simply indicates 



13 Respondents bom in Nebraska were omitted since that state does not use Carnegie units in defining its 
graduation standards. I identified the year in which each respondent was 18 by their age on enumeration 
day (April 1, 1990). 

14 College entrants are those whose highest reported educational attainment was "Some college, no degree" 
or higher. Unfortunately, PUMS does not distinguish high school graduates from GED completers. 
However, to the extent that higher standards generated a reduction in this measure of high school 
completion, we can conclude there was not a completely offsetting increase in GED completion. It should 
also be noted that this sample, of course, includes students who attended private schools. However, their 
inclusion is arguably appropriate since it is possible that students may switch schools to avoid the 
consequences of stricter standards. 

15 Those who report that they are not in the labor force are defined as unemployed to avoid omitting 
discouraged workers. However, the exclusion of these respondents does not substantively alter the 
subsequent results. 

16 The school attendance variable is meant to control for the fact that those respondents still in school over 
the last year would have had limited labor market experiences. This specification is similar to those used 
by Bishop and Mane (2001). However, I also estimated purer "reduced-form" versions of this model that 
excluded the measures of educational attainment and school attendance. The results were similar to those 
reported here (Table 4). 

17 1 report Huber- White heteroscedastic-consistent standard errors, which assume clustered effects specific 
to each state-of-birth by year-of-birth cell. Also, I found that probit models for the binary dependent 
variables generated similar results. 
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whether a MCT was required for that particular graduating class (Table I). The second dummy identifies 
whether the state had a high, academically focused CGR in effect for that graduating class. A high CGR is 
defined here as a required high school curriculum that includes at least 3 Carnegie units in English, 2 in 
social studies, 1 in science and 1 in mathematics. 18 These and other state-year controls were matched to the 
respondents by their state of birth and year of birth. One noteworthy limitation of the PUMS data is that 
relying on state of birth may introduce measurement error since some children have moved to different 
states by the time they reached high school. However, the attenuation bias implied by such measurement 
error suggests that the reported estimates can be interpreted as lower bounds on the true effects. 19 

As suggested earlier, the identification strategy embedded in this model makes a potentially 
important contribution to our understanding of the consequences of first-wave reforms because it removes 
the possible biases due to unobserved state-level determinants. This model effectively does this by 
comparing the cohort differences in the ’'treatment" states before and after the introduction of new 
standards to the contemporaneous cross-cohort changes in the ’’control" states. I present some evidence on 
the empirical relevance of relying on within-state versus cross-state comparisons by comparing the results 
of models that do and do not include the state fixed effects, p s . I also present some heuristic evidence on 
this specification issue through the use of a simple counterfactual in which I estimate the “effect” of a state 
policy that should not have large and statistically significant effects on educational attainment. To the 
extent that a particular specification suggests that this policy did have large and statistically significant 
effects, the existence of specification error is suggested. 20 

In the preferred specifications, which include state fixed effects, the possible sources of omitted 
variable biases are the unobserved determinants of Y that are also related to the timing of new standards 
within states. The matrix, Z st , addresses this concern by including other regression controls that vary by 
state and year. For example, new state standards were sometimes part of omnibus education bills that 
included other policy changes such as increased spending. To control for the possible effects of school 
spending, some models include, as an independent state-level variable, real expenditures on K-12 public 
schools per student in average daily attendance when the respondents were 16 to 17 years old. For 
example, respondents who were 18 in 1980 were matched to the school expenditures in their state during 
the 1978-1979 school year. Another state-year control in most models is the state unemployment rate when 
the respondent was 17 years old. This variable is expected to have a positive effect on educational 
attainment since it reduces the opportunity costs associated with remaining in school (Duncan 1965). Card 



18 Some studies represent state CGR policies by the total number of Carnegie units required. However, this 
measure may more accurately reflect the focus of reform efforts (e.g., A Nation at Risk), which was to 
course taking in core academic areas. 

19 

Also, I found that the results were similar in models that matched respondents to the state-year variables 
by their state of residence five years prior to the Census. 

20 Specifically, I estimate, in models with and without state fixed effects, the ’’effects’’ on educational 
attainment of whether there were any state executions at age 18. One virtue of using state executions for 
this counterfactual is that there was considerable variation over this period both within states and across 
states. An increasing number of states began executions in 1984. 
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and Lemieux (2001) present evidence that the natural variation in the size of particular birth cohort’s 
population can also influence educational attainment. At the college level, this could occur if temporary 
increases in cohort size are not fully matched by an increased supply of enrollment space at local colleges 
and universities. At the secondary level, increased cohort size may reduce the benefits of remaining in 
school by lowering school quality. 21 Therefore, I also include a measure of cohort size based on the natural 
log of the U.S. Census Bureau's estimate of 18 year-olds in the respondent's state of birth at age 18. I also 
include a measure of the real costs of postsecondary tuition based on the in-state rate at "lower-level" state 
colleges and universities when the respondent was 17 years old (Card and Lemieux 2001, Kane 1994). 
Finally, as a control for within-state changes in socioeconomic priors, I also matched each respondent to 
the poverty rate in their state when they were 17 years old. 

RESULTS 

In Table 2, I present the estimated effects of MCT and CGR policies on educational attainment 
across a variety of specifications. These results demonstrate the estimated effects of the first-wave reforms 
on educational attainment are quite sensitive to controlling for unobserved state fixed effects. For example, 
the models that exclude state fixed effects but include the other state-year controls suggest that MCT 
significantly reduced the probability of graduating high school and attending college. These results also 
suggest that CGR policies had no statistically significant effects on either measure of educational 
attainment. However, the models that include state fixed effects and the other state-year controls, imply 
that MCT had small and statistically insignificant effects on both outcomes. These models also suggest that 

higher CGR reduced the probability of graduating high school by a statistically significant 0.48 percentage 

22 

points. 

Several dimensions of these results in Table 2 suggest that the inferences from the models with 
state fixed effects are more reliable. First, F-tests indicate that the state fixed effects are jointly significant 
determinants of educational attainment. Second, in models that exclude state fixed effects, the key results 
are highly sensitive to the presence of the other state-year controls. For example, similar to Bishop and 
Mane (2001), the first model implies that MCT significantly increased the probability of attending college. 
However, after introducing the other controls, this estimate becomes negative and significant. This type of 
sensitivity suggests the difficulty of relying on proxies for the determinants of educational achievement 
across states. Third, the sensitivity of these evaluation results to the introduction of state fixed effects does 
not appear to reflect any loss of sampling variation or statistical precision. Specifically, some of the cross- 
state models suggest that MCT and CGR had effects roughly 2 percentage points in size. However, in the 



21 Card and Lemieux (2001) find that cohort size is associated with significant increases in pupil-teacher 
ratios. 

22 An effect of this size represents a 0.6 percent reduction in the mean probability of graduating from high 
school, or, alternatively, a 3 percent increase in the mean probability of dropping out. 
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preferred specification (column 5 of Table 2), the standard errors are sufficiently small that effects of that 
size can be rejected at conventional levels of statistical significance. 

A final way to provide some ad-hoc evidence on the reliability of cross-state versus within-state 
comparisons is through the use of a simple counterfactual. To the extent that empirical evaluations relying 
on cross-state comparisons generate reliable results, we should find that, conditional on the other controls, 
irrelevant state policies have small and statistically insignificant effects on educational attainment. 

However, to the extent that an irrelevant policy appears to have a large and statistically significant effect, it 
suggests that the existence of biases driven by the unobserved, state-specific determinants of educational 
outcomes. The results in Table 2 present such evidence by reporting the estimated "effects" on educational 
attainment of having any state executions at age 18. 23 The models without state fixed effects suggest that 
capital punishment generates large and statistically significant reductions in the probability of high school 
completion (1.3 percentage points) and college entrance (1.7 percentage points). However, in the models 
that rely on the within-state variation in executions, these estimates are much smaller, more precisely 
estimated and statistically insignificant. 

In Table 3, I present the key evaluation results from the preferred specifications that include both 
the state-year fixed effects and the state-year controls. I also report the estimated effects of these first-wave 
reforms from separate models for white males, white females, black males and black females. All of these 
models suggest that the first-wave reforms had statistically insignificant effects on the probability of 
entering college. However, these results also indicate that these reforms had fairly large and statistically 
significant effects on the probability of completing high school and that these effects varied considerably 
by race and gender. In particular, these estimates suggest that higher CGR significantly reduced the 
probability of completing high school for white males and blacks but not for white females. 24 Notably, the 
reform-driven reductions in educational attainment were particularly large among blacks (roughly 2 
percentage points). These estimated reductions are roughly four times larger than those for white males. 
Similarly, these results suggest that the only large and statistically significant effect of introducing MCT 
was among black males who experienced an estimated 1.26 percentage point reduction in the probability of 
completing high school. 

The evidence from Table 3 is largely consistent with the concerns sometimes raised by critics of 
standards- based reform (e.g., Mumane and Levy 2001). The introduction of high school graduation 
standards led to reductions in educational attainment that were particularly concentrated among black 
students. However, a full evaluation should also consider the implications of these reforms for labor 
market experiences. Attention to the labor market consequences of these policies also has a strong intuitive 



23 The execution of criminals could have an actual influence on educational attainment since the 
prosecution of these cases can be a meaningful drain on public resources. However, it should be noted that 
this reduced-form effect should be quite small and that these models control for real school spending per 
pupil. 

24 It should be recalled that the PUMS definition of a high school graduate includes GED completers so 
these results may actually understate the true reform-induced reduction in high school completion. 
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appeal since local business leaders concerned with the quality of their work force were often instrumental 
in the adoption of first-wave reforms. As noted earlier, higher standards may benefit students (even those 
who drop out) by inducing increased educational effort that is rewarded in the labor market (an "incentive 
effect"). There may also be distributional consequences of these reforms to the extent that higher standards 
increase the prestige of being a high school graduate and correspondingly reduce the stigma associated with 
being a dropout (a "sorting effect"). 

Table 4 presents new evidence on these issues by reporting the estimated effects of the first-wave 
reforms on employment participation and log wages for the full PUMS sample and for samples defined by 
race and gender. These models include state and year fixed effects, the state-year controls and additional 
individual-level controls for educational attainment and student status. Unlike the prior cross-sectional 
evaluations, these results suggest that both reforms had small and statistically significant effects on wages 
for all groups. However, these results also suggest that the first-wave reforms had statistically significant 
effects on employment participation. Specifically, these estimates suggest that higher CGR increased the 
probability of being employed by roughly 1 to 3 percentage points for white males and blacks. As with the 
model for high school completion, higher CGR had small and statistically insignificant effects for white 
females." The introduction of MCT significantly increased the probability of employment for black males 
(1.64 percentage points). 

The results in Tables 3 and 4 suggest that new CGR were a meaningfully binding standard that 
had educational and labor market consequences for almost all students. In contrast, the effects of MCT 
were more limited. These results are consistent with the anecdotal evidence suggesting that MCT were 
"implemented as a lamb" in response to political realities. These results also suggest that, when binding, 
higher standards of either type had decidedly mixed distributional consequences. They reduced educational 
attainment, particularly among black students. However, they also generated some labor market rewards in 
the form of increased employment probabilities that were concentrated among black students. How can we 
compare these gains and losses? One possibly useful point of reference is the expected wage associated 
with being a high school graduate or a dropout. A rough calculation based on these data suggests that high 
school graduates receive an expected wage premium equal to approximately 33 percent of a dropout's 
average wage, wa. 26 This implies that those who dropped out of school in response to the higher standards 
suffered substantive consequences. Their loss of this wage premium was offset only somewhat by a .0081 
increase in the probability of employment as a dropout. However, for those who would have dropped out 



25 However, MCT significantly reduced the employment probability for white females by 0.95 percentage 
points. Since MCT did not significantly influence educational attainment among white females, this 
reduction could reflect a lowering of educational effort. The results in the next section provide some 
suggestive evidence that this may be so by evaluating the effects of these reforms on individual "process" 
measures drawn from transcript data and survey questions. 

26 The wage regression indicates that the wages of high school graduates are 18 percent higher than those of 
dropouts. The mean probability of employment among dropouts is 54 percent. The regression model for 
employment participation suggests that high school graduates are 20 percentage points more likely to be 
employed. Therefore, the implied premium in expected wages is 0.33 wa (i.e., .74 x 1.18- .54). 
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or graduated without regard to the changed graduation requirements, there were unambiguous labor-market 
gains since they are significantly more likely to be employed. 

Another possible useful way to frame these costs and benefits is to ask how these reforms might 
change the expected wage for someone who was uncertain about whether they would be a high school 
graduate or not. For such a person, the expected cost of higher CGR is the reduction in the probability of 
enjoying the 33 percent wage premium of high school graduates. This equals .001 6w d (i.e., .0048 x .33). 
The benefit of a higher CGR is an increased probability of being employed (.0081) at an expected wage 

equal to 1. 155w d (i.e., .86 x 1.18 + .14). 27 This expected benefit equals .0094w d . Therefore, the expected 

28 

wage benefits of a higher CGR exceed expected wage costs by a factor of roughly 6 (.0094/.001 6). This 
suggests that a risk-neutral person might prefer a regime with higher standards to one without and that the 
net effects of the higher standards on expected wages are positive. However, it should also be noted that 
these back-of-the-envelope calculations do not constitute a full cost-benefit analysis. In particular, these 
comparisons ignore the other social losses that may be associated with reform-induced reductions in 
educational attainment (e.g., those related to health, criminal and civic behaviors). 

5. STANDARDS AND EDUCATIONAL PROCESSES 

The evidence from the prior section suggests that first-wave reforms sometimes reduced 
educational attainment but also generated some improvements in the probability of employment. These 
labor market consequences of stricter graduation standards could, in most cases, simply reflect passive 
sorting effects (Betts and Costrell 2001). However, they could also indicate reform- induced increases in 
educational effort, which were subsequently rewarded in the labor market (i.e., incentive effects). In this 
section, I provide some empirical evidence on the second possibility by examining how the first-wave 
reforms influenced several educational "process" measures. Most of these process measures reflect patterns 
in academic course-taking among individual high school students. Interestingly, the amount of academic 
course-taking among public high school graduates did increase significantly during the 1980s across 
students of varying demographic traits (U.S. Department of Education 2001a, Table 138). These increases 
were particularly large in mathematics and science. For example, the average number of Carnegie units 
among public high school graduates in 1982 and 1994 increased 27 percent in mathematics and 38 percent 
in science (U.S. Department of Education 2001a). Some studies have suggested that the new CGR were at 
least partially responsible for these increases. For example, the U.S. Department of Education (1997) finds 
that the graduates in high-CGR states have higher levels of academic course-taking. Similarly, Clune and 
White (1992), in a study of four states, found that academic course-taking among graduates increased after 



27 

Given the stated uncertainty about whether the observer will be a dropout or not, the expected wage is 
based on the probabilities of being a graduate or a dropout (.86 and .14) and the wage premium for 
graduates (18 percent). However, these calculations generate similar results if we assume the observer only 

receives w d . 

28 

For black males and females, the policy-induced gains in expected wages are also several times larger 
than the corresponding losses. 
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the introduction of higher CGR. However, there are two specification issues that may bias these inferences 
about the effectiveness of first-wave reforms. One is that the appearance of policy-induced increases in 
academic course-taking could simply be due to the increased dropout rate (Table 3) instead of genuine 
increases in academic effort. A second concern is that the identification strategies, which rely exclusively 
on either cross-state or time-series comparisons, may lead to substantively biased inferences. 29 The 
evaluations presented here provide new evidence on these issues by examining estimates from models that 
include eventual dropouts and that control for unobserved state and year fixed effects. 

In this section, I also present new evidence on concerns raised by critics of stand ards-based reform 
that higher standards may have unintended and pejorative effects on other educational processes. For 
example, one concern is that creating minimum standards may encourage high-performing students to 
reduce their academic effort. I present evidence on this issue by evaluating how first-wave reforms 
influenced high-level course taking in math and science (i.e., Carnegie units in calculus, physics and 
chemistry). I also present evidence on whether first-wave reforms narrowed student curricula by reducing 
Carnegie units in the visual and performing arts. Another related concern about establishing explicit 
standards for students is that they may discourage intellectual engagement and lifelong learning by 
suggesting that learning for its own sake is not worthwhile. While measuring these sorts of attitudes is 
inherently difficult, the U.S. Department of Education (2001b), in its annual report. The Condition of 
Education , suggests that students' use of time is closely related to their educational aspirations and feelings 
about school. They also note that, between 1984 and 1999, 17-year olds report watching less television. 
However, these 17-year olds are also less likely to read for pleasure and spend less time doing homework 
(in part, because it is less likely to be assigned). In this section, I also present evidence on how first-wave 
reforms may have influenced the amount of time students spend on these three activities. 

DATA AND SPECIFICATIONS 

The data for these evaluations were created by pooling observations from two of the National 
Center for Education Statistics' (NCES) major longitudinal studies: the sophomore cohort from High 
School and Beyond (HS&B) and the National Education Longitudinal Study of 1988 (NELS). These 
surveys provide student-level data from before and after the time when most first-wave reforms were 
implemented (Table 1). More specifically, HS&B and NELS provide nationally representative samples of 
10 th grade students from 1980 and 1990, respectively. 30 Since each of these studies had a transcript 
component, they also include data on the Carnegie units earned in particular subject areas in addition to 



29 For example, a pre-reform/post-reform time-series comparison may attribute to higher CGR increases in 
academic course-taking that are actually due to other time-varying determinants (e.g., the growing college 
wage premium in the 1980s). 

30 My NELS extract does not include all NELS respondents since some of the base-year sample (i.e., those 
in 8 th grade in 1988) were not in 10th grade at the time of the 1990 follow-up. However, in order to be 
representative of high school sophomores, the extract does include respondents who "freshened" the 
sample: 10 th graders who were not in 8 th grade in 1988. 
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survey questions on students’ use of time. 31 My extract from these surveys consisted of white non-Hispanic 
and black respondents who were 10 lh graders in 1980 (HS&B) and 1990 (NELS) and includes eventual 
dropouts. The combined sample size with available transcript data consists of 1 8, 134 observations (9,33 1 
from HS&B and 8,803 from NELS). 32 

The econometric specification I used for models based on these data is similar to the preferred 
specification from the previous section. The independent variables of interest reflect the state high school 
graduation requirements in effect for the graduating classes of 1982 and 1992 respectively (Table 1). The 
other independent variables include state and year fixed effects, individual-specific variables and variables 
specific to each state-year cell. The individual-level controls include single dummy variables for race, 
gender and age (bom before 1964 for HS&B respondents and before 1974 for NELS respondents). These 
controls also include four dummy variables for the highest level of parental education, five dummy 
variables for family composition and four dummy variables for SES quartile (including one for a missing 
SES index). I matched each respondent to the relevant graduation requirements and other state-year 
controls by exploiting the state identifiers in the restricted-use versions of these surveys. 33 The state-year 
controls again include 1981 and 1991 data on real public school spending per capita, the state 
unemployment rate, the poverty rate and the real postsecondary tuition level. I also matched each 
respondent to the size of their state-year cohort: 1982 and 1992 data on the natural log of the 18-year old 
population in their state. 

RESULTS 

In Table 5, I present estimates of how first-wave reforms influenced the amount of academic credit 
earned in the four core subject areas. These results uniformly suggest that the introduction of MCT 
reduced course taking in these academic areas. However, these estimated reductions are statistically 
significant only in the sciences and mathematics. The estimated reductions in science and mathematics 
attributable to MCT are roughly equal to 5 percent of the dependent means. In contrast, the evidence in 
Table 5 suggests that higher CGR had the desired effect of generating substantive increases in the credits 
earned in these core areas. In particular, a high CGR (3 or more Carnegie units required in the subject) led 



Each course taken by a student was assigned one of the more than 2,200 unique codes associated with the 
Classification of Secondary School Courses (CSSC). For details of each transcript study, see U.S. 
Department of Education (1995, 1997). 

32 The sample sizes for the non-transcript outcomes are over 19,000. The lower sample size for course- 
taking data largely reflects the fact that the schools of some NELS respondents were not surveyed for 
transcripts (U.S. Department of Education 1995). Respondents from Nebraska were also excluded because 
the state does not use Carnegie units in defining its graduation standards. 

33 Interestingly, the restricted use version of HS&B does not directly identify each high school students’ 
state. Lillard and DeCicca (2001, footnote 13) imputed the state of each HS&B school by identifying the 
state in which its graduates attended community college. I identified the states of HS&B schools through a 
cross-walk of the HS&B data on the 1980, 1981 and 1982 state unemployment rates and published 
numbers. 
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to increased credit in each academic subject. 34 For example, a high CGR in science increased credits 
earned by 0.393 relative to the reference category of a weak or non-existent CGR (less than 1 Carnegie unit 
required in subject). This estimated effect is roughly equal to 16 percent of the mean science credits. 
However, the estimated effect of a high CGR on social studies was statistically insignificant as were the 
much smaller estimated effects of weaker CGR (1-2.99 Carnegie units required). Nonetheless, these results 
suggest that new CGR did contribute substantively to the academic upgrading of high school curricula over 
this period, particularly in English and the sciences. For example, the estimated effect associated with a 
high CGR in science is equal to roughly 60 percent of the average growth in science credits over this 
period. 

The results in Table 5 suggest that the introduction of an MCT led to a reduction in curricular effort. 
However, there is some ambiguity to these results since it is unclear to what extent these reductions are 
driven by students who may be induced into dropping out (Table 3) or by high-performing students who 
choose only to meet the new minimum requirements. In Table 6, 1 provide some indirect evidence on these 
and other issues by reporting the estimated effects of these reforms, by race and gender, on Carnegie units 
earned in science and mathematics. The results indicate that MCT led to particularly large reductions in 
science and mathematics credits for black students. Since black males were also more likely to drop out 
because of MCT (Table 3), the policy-induced reductions in credits earned may be due in part to an 
increased intent to drop out. However, the introduction of MCT also led to particularly reduction in math 
and science credits for female students. These reductions are more likely to reflect lower levels of 
curricular effort among otherwise high-performing students since there is no indication that females were 
more likely to drop out because of a new MCT (Table 3). Interestingly, the results in Table 6 also indicate 
that high CGR in mathematics and science had particularly large and positive effects on the academic 

35 

credits earned by black students, substantially reducing their gap relative to white students. 

The results in Tables 5 and 6 suggest that MCT sometimes had negative effects on curricular effort 
while the effects of higher CGR were often positive. One of the difficulties of interpreting those results is 
that they define the policies' effects for the "average” student. However, it is reasonable to suspect that 
these reforms could have unique effects on low-performing students. In Table 7, 1 present alternative 
evidence on how first-wave reforms influenced the amount of academic credits earned at a low-performing 
margin. More specifically, I constructed new dependent variables: dummy variables equal to 1 for students 
who earned at least one Carnegie in a given academic subject. Across the four academic subjects, 95 to 98 
percent of students met this weak standard. In Tables 7 and 8, I present regression results from models 
whether these indicators are the dependent variables. 



34 The reference category is a weak or non-existent CGR (less than 1 Carnegie unit required in subject). 

35 Cook and Evans (2000) find that only 25 percent of the convergence in black-white test scores is 
attributable to changes in family background while 75 percent reflects changes within schools. The relative 
academic upgrading generated for black students by higher CGR may be one of the within-school changes 
responsible for this convergence. 
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The regression results in Table 7 suggest that, in each academic subject, the introduction of MCT 
reduced the probability that students would meet this low standard. However, these estimates are estimated 
imprecisely and most are statistically indistinguishable from zero. In contrast, these results also suggest 
that higher CGR sometimes increased the probability of having at least one Carnegie unit in these academic 
subjects. In particular, these estimated effects were, again, relatively large and statistically significant in 
English and the sciences. These results are striking since they suggest that, while higher CGR may have 
encouraged more students to drop out (Table 3), they may have also upgraded their academic curricula 
prior to dropping out. This implies that any subsequent labor market gains associated with higher CGR 
could reflect incentive effects as well as sorting effects. In Table 8, 1 present the results for science and 
mathematics in race-specific and gender-specific models. These results indicate that higher CGR increased 
the probabilities that students from each demographic group would have at least one Carnegie unit in the 
sciences. Notably, these policy-induced increases in science credits at this margin were particularly large 
for black and female students. 

In Table 9, 1 present some evidence on how these reforms influenced the amount of credit earned 
in advanced courses in mathematics (i.e., calculus) and the sciences (i.e., physics and astronomy). 
Interestingly, these regressions suggest that the introduction of MCT led to a large and statistically 
significant reduction in calculus credits but had smaller and statistically insignificant effects on physics and 
chemistry credits. These results also suggest that an increased CGR in science led to large and statistically 
significant increases in physics credits but not in chemistry credits. The results in Table 9 also indicate that 
a high CGR led to a weakly significant increase in calculus credits. Overall, like the results in Tables 5 
through 8, these results provide fairly consistent evidence on how first-wave reforms influenced student 
curricula in core academic areas. More specifically, these results indicate that the introduction of MCT led 
to some reductions in curricular effort among students likely to meet the basic testing requirement. 

However, these results also indicate that higher CGR made substantive contributions to the upgrading of 
academic curricula over this period and did so for students at very different margins. 

However, other concerns about student-level standards involve whether they narrow student 
curricula in undesirable ways or reduce the intellectual engagement of students. I present some evidence 
on these issues in Table 10. First, I report the estimated effects of the first-wave reforms on student 
involvement in the visual and performing arts (as measured by Carnegie units) and on student participation 
in school musical activities (i.e., the school band, orchestra or chorus). For both outcomes, the estimated 
effects of the first-wave reforms were imprecisely estimated and statistically indistinguishable from zero. 
The final regression results in Table 10 focus on indirect measures of intellectual engagement and effort 
based on the students' reported use of time. Specifically, I created dummy variables for whether the 
respondent never or rarely reads for pleasure outside of school, for whether they watch four or more hours 
of television per weekday and for whether they do no homework/ 6 The results in Table 10 suggest that the 



j6 The HS&B and NELS questions on time spent doing homework were not consistently phrased. The 
HS&B question (BB015) allowed students to distinguish between doing no homework and having none 
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introduction of MCT led to a large but weakly significant increase in the probability of doing no homework 
(t-statistic = 1.95). However, the estimated effects of a high CGR (i.e., a "3/2/1/1" standard or higher) on 
students' use of time were larger and more precisely estimated. More specifically, the results in Table 10 
indicate that a higher CGR led to large and statistically significant reductions in the amount of time 
students spend reading for pleasure and doing homework and a correspondingly large increase in time spent 
watching television. It should be noted that there is some ambiguity in interpreting these results. It could 
reflect the direct effect of imposing minimum requirements on student perceptions of the value of learning. 
However, there may also be other issues related to the implementation of new CGR that contribute to these 
results as well. For example, it may be that less homework was assigned after the increases in state CGR. 
Nonetheless, these results suggest that CGR-induced increases in academic course taking were attenuated 
by undesirable changes in how students allocate their time. 

6. LESSONS FROM THE FIRST WAVE 

Our ongoing debate about the design and desirability of standards-based reform hinges critically 
on how such policies may influence a variety of outcomes among students with different backgrounds. In 
this paper, I provided new evidence on those issues by examining the effects of the earlier state-level 
standards on several outcome and process measures. These results demonstrated that the first wave of 
student-level standards appear to have had many of both the positive and negative effects suggested by 
commentators on both sides of these issues. For example, these reforms led to reductions in educational 
attainment that were particularly large for black students. Furthermore, minimum competency testing 
(MCT) led to some apparent reductions in curricular effort while higher course graduation requirements 
(CGR) had pejorative effects on the amounts of time students spend watching television, doing homework 
and reading for pleasure. However, these reforms also increased subsequent employment probabilities. 

And higher CGR were partly responsible for the substantial academic upgrading of high school curricula 
that occurred over this period. In light of this mixed evidence, what can these prior state-level experiences 
contribute to our current discussions about standards? 

A productive, though modest, initial step may be to consider what these results would suggest, to a 
proponent of standards-based reform, about how those standards should be designed. In particular, the 
first-wave reforms provide an interesting basis for comparison since they included both a test-based 
standard (MCT) and a "process" standard (CGR). The results presented here suggest that advocates of 
standards-based reform may prefer the ultimate effects of process standards to those of a test-based 
standard. More specifically, minimum competency testing had relatively few of the desired effects on 
educational attainment and early labor market experiences (Tables 3 and 4). These results are consistent 
with the widely held perception that these test-based standards were often quite weak because of political 
pressures and the relatively easy and veiled manner in which they could be subsequently lowered. In 

assigned but did not distinguish between homework done in and outside school. My coding of this variable 
is based on this HS&B question and the NELS question (F1S36A2) for homework done outside of school. 
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contrast, newly introduced course graduation requirements (CGR) created more binding, new standards for 
students (Tables 3 and 4) and they were also largely immune to subsequent political redesign. 

The evidence from student-level transcripts provides additional support for the relative 
attractiveness of process standards. More specifically, the results in Tables 5 through 9 suggest that CGR 
contributed directly to the academic upgrading of the high school curricula of a wide variety of students. In 
contrast, this evidence also suggests that some students actually lowered their curricular effort, particularly 
in the sciences and mathematics, in response to the introduction of MCT. The one caveat to the 
comparative attractiveness of CGR is that their benefits may be attenuated by changes in teacher 
expectations (e.g., how much homework is assigned) and changes in how students allocate their time 
(Table 10). Furthermore, whether these comparative, first-wave results have much "external validity” for 
ongoing efforts to develop test-based accountability is clearly open to conjecture. But, at a minimum, the 
early state-level experiences with minimum competency testing provide an important, cautionary tale. 

The implications of the results presented here for the broader debate over whether standards are 
desirable type of education reform must be based on more subjectively normative grounds. For example, 
an advocate of a Rawlsian social welfare function (Rawls 1971) would almost certainly look with favor on 
such reforms since they increase the employment probability of those who would have dropped out of 
school anyway. However, others with a more utilitarian perspective may be less willing to accept small 
employment gains for many students at the cost of significant welfare losses among those encouraged by 
new standards to drop out of high school. It is exactly those welfare losses that encourage advocates of 
standards- based reform to recommend the simultaneous adoption of targeted efforts to assist those who 
may be newly at risk of dropping out (e.g., Betts and Costrell 2001). Similarly, critics of standards also 
suggest that, if standards are to be implemented, they should be accompanied by increased "capacity 
building" in the form of higher teacher salaries, teacher training and local control of schools (e.g., Mumane 
and Levy 2001). 

However, such approaches to attenuating the difficult trade-offs implied by higher standards may 
provide a deceptively facile solution. In particular, it should be recalled that a fundamental motivation for 
instituting standards in the first place has been the controversial claim that educational inputs cannot be 
targeted in ways that systematically promote student achievement. So, recommendations to help somehow 
the students harmed by standards bring us back to the notoriously difficult research questions about which 
programs or expenditures might actually be effective. 

The experiences within some of the states that adopted first wave reforms suggest we should not 
be too sanguine about their ability (or willingness) to craft solutions that soften these difficult trade-offs. 

For example, we can consider the context in which first wave reforms were introduced in the two states 
with the largest public school enrollments. The state of California instituted a new course graduation 
requirement (first effective for the graduating class of 1987) as part of the Hughes-Hart Educational 
Reform Act of 1983 (Senate Bill 813). This legislation was a comprehensive school reform package that 

However, I found that alternative coding schemes generated similar results. 
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combined a new state CGR with $800 million in new funds targeted at over 80 other initiatives including 
higher starting salaries for teachers and a teacher mentoring program (Cuban 1984, Pipho 1986). Similarly, 
the state of Texas introduced minimum competency testing in 1984 (first effective for the graduating class 
of 1987) as one component of an extensive package of school reforms (House Bill 72). These reforms 
included a variety of other complementary initiatives such as increased starting salaries for teachers, a 
teaching career ladder, management training for principals and superintendents and a "no pass, no play" 
restriction on extracurricular activities (Pipho 1986). Some districts in Texas also responded to the MCT 
by developing summer-school initiatives targeted at those at risk for dropping out because of the new 
standards (Archer and Dresden 1987). These examples indicate that many of the students who faced new 
state-level graduation standards were also supported by a contemporaneous mix of other financial and 
regulatory changes. This implies that the difficult trade-offs indicated by this study’s empirical results are a 
relatively intractable feature of introducing higher student-level standards. This interpretation suggests that 
our ongoing public discussions about the desirability of highly centralized standards should explicitly 
address how we might value those trade-offs. Furthermore, these discussions should also consider how the 
diverse set of policy effects presented here might compare to those of alternative proposals such as the 
second wave reforms that stress decentralization and local control. 
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Table 1 - State High School Graduation Requirements, 1980 and 1992 



State 



Minimum 

Competency Test (MCT) 



Course Graduation Requirements (CGR) 



1980 



1992 



V C J 






Alabama 
' JAlaskav ^ ~ 
Arizona 

California 
- Color^o;|||Pl|! 

Connecticut 
: Delaware gg™ 
Florida 
Georgia 
Hawaii 

, idahd^lllJIIIIIJI 

Illinois 

IflndiM 

Iowa 

||Rans® 

Kentucky 

Maine 

: Maryland J|J7f|J 
Massachusetts 

Minnesota 

Mississippi 

Missouri 

Montana 

Nevada 

New Hampshire 
New Jersey 
i^New Mexico 
New York 
North Carolina 
North Dakota 
Ohio 

Oklahoma 

vl^Oregdmlijl^iJ 

Pennsylvania 
Rhode Island 
South Carolina 



1985 



4/3/ 1/1 
J/I/l/1 
3/2/ 1/1 
4/1/0/0 



4/3/2Z2 

! ■ ■ . 4/3/2/2'i 

4/2. 5/2/2 

r ' :: ' 4/3/2:5/15' 

3/3/2/2 






4/3 /2/3 

1983 " O/O/O/O ~ 413/3/3 

1985 f 3/1/1 /l 4/312/2 ' / 

1983 — — 

- ^ ' 3/i/O/O 3/2/ 1/2 

, 6/i .5/676 






gisupHsuas 



. V, vs ,s 









i 






3/2/1/2 4/2/2Z3 

III 

"77" * "" " 6/ l/o/o 

^ — - '^"4/3/t/i ' 

l/l/l/l 3/2/2/2 

iliSSI 



lliill 



Miim 



1982 


3/2/ 1/1 


4/2/2/2 




^ :>.V ..v..-. . . ^ 




4/2.5/212 




1981 


4/2/ 1/2 


473/2/3 




„ .. 

«■ v ^ s , v ' * < J&m*. _ « > : ™ 


Sillllll; 


|j§§ 


1979 


4/3/1/1 


" 4 / 4 / 2/2 




1980 

svvw^s 


MHBSHH 


4/2/2Z2 


& 


- 


'" ~ 3/3/2/ 1 


473/2/2 






| §|§ Jgj Jlllllll ; j;3/2/i||| |!| ; | 1 1||| || ||i||||l|; ;| 


: 3/2/ 1/2 : ^ 











i 



4/1. 5/1/1 4/2/2/2 
3/4/1 /l 3/3. 5/2/2 
3/2/ 1/1 4/3/3 /3 
4/1/1/1 4/2/2Z2 



1990 



4/3/ 1/2 



4/3/213 
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South Dakota 
Tennessee 



' fZ\ 



Utah 

IKgerrnont 
Virginia 
jpV ^hingi onjpiC i 
West Virginia 
Wisconsin 
Wyoming 









4/2/ l/l 4/3/3/2 

1982 4/1. 5/1/1 * “ 4/ 1/2/2 

1987 3/2. 5/2/2 4Z3/2/3 



: y - <• • 



3/2/ 1/1 



4/3/ i /I 



|i|l _ 



|i|#5;6/5/2/3 : ; 

4/3/ i/P' 

o/ i/o/6 



lliiii 



3/3/2/2 
4/3/2.5/25 
473/2/2 
3/2. 5/2/2 
4/3/2/2 
4/3Z2/2 
o/i/o/o 






The year in the "Minimum Competency Test" column refers to the first graduating class for which the 
requirement was effect. The course requirements indicate the Carnegie units required for the graduating 
classes of 1980 and 1992 in English, Social Studies, Science and Mathematics, respectively. Nebraska was 
excluded because it does not use Carnegie units. The sources for these data include various publications 
from the Education Commission of the States, Jacob (forthcoming) and state-specific searches of 
newspaper articles. 
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Table 2 - Linear Probability Models for Educational Attainment, 1990 PUMS 



Independent Variable Estimated Effects 



High School Graduate 



Minimum competency test (MCT) 


.0001 

(.0065) 


-0187J 

(.0033) 


-.0165J 

(.0031) 


-.0013 

(.0020) 


-.0014 

(.0020) 


-.0015 

(.0020) 


High course graduation requirement 


-.0051 


-.0027 


-.0020 


-.0055f 


-.0048f 


-.0048f 


(CGR) 


(.0041) 


(.0024) 


(.0023) 


(.0025) 


(.0022) 


(.0022) 


Any state executions at age 18? 


- 


- 


-.01261 

(.0041) 


- 


- 


.0004 

(.0019) 








College Entrant 






Minimum competency test (MCT) 


.0244f 

(.0113) 


-.0240J 

(.0056) 


-.021 1 1 
(.0054) 


-.0054* * 

(.0028) 


.0002 

(.0030) 


.0009 

(.0030) 


High course graduation requirement 


-.0197J 


-.0058 


-.0047 


-.0079* 


-.0028 


-.0028 


(CGR) 


(.0075) 


(.0046) 


(.0045) 


(.0048) 


(.0033) 


(.0033) 


Any state executions at age 18? 


- 


- 


-.0 1 70 J 
(.0064) 


- 


- 


-.0026 

(.0024) 


State Fixed Effects? 


No 


No 


No 


Yes 


Yes 


Yes 


State-Year Controls? 


No 


Yes 


Yes 


No 


Yes 


Yes 



These results are based on the 1,348,766 white non-Hispanic and black respondents who were age 18 
between 1980 and 1988. All the models include fixed effects for race, gender, and year of birth. The state- 
year controls are the unemployment rate, the natural log of cohort size, poverty rate, real K-12 expenditures 
per-pupil and real post-secondary tuition. Heteroscedastic-consi stent standard errors, adjusted for state-of- 
birth by year-of-birth clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
f Statistically significant at 5-percent level 
{ Statistically significant at 1 -percent level 
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Table 3 - Estimated Effects of Minimum Competency Tests (MCT) and High Course Graduation 
Requirements (CGR) on Educational Attainment by Race and Gender, 1990 PUMS 







Dependent Variable 








High School 
Graduate 


College 

Entrant 


Sample 


Sample 


MCT 


CGR 


MCT 


CGR 


Size 


All respondents 


-.0014 

(.0020) 


-.0048| 

(.0022) 


.0002 

(.0030) 


-.0028 

(.0033) 


1,348,766 


White Males 


.0025 

(.0033) 


-.00531* 

(.0025) 


.0049 

(.0045) 


-.0026 

(.0040) 


585,376 


White Females 


-.0022 

(.0026) 


-.0005 

(.0032) 


.0007 

(.0035) 


-.0017 

(.0035) 


588,611 


Black Males 


-.01261* 

(.0055) 


-.021 It 
(.0087) 


-.0024 

(.0073) 


-.0161 

(.0111) 


81,799 


Black Females 


.0010 

(.0054) 


-.0203 X 
(.0070) 


.0044 

(.0070) 


-.0188 

(.0015) 


92,980 



These estimates are based on linear probability models, which include fixed effects for race, gender, state 
of birth and year of birth as well as the controls for the unemployment rate, cohort size, poverty rate, real 
K-12 expenditures per-pupil, real post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-of-birth by year-of-birth clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
f Statistically significant at 5-percent level 
% Statistically significant at 1 -percent level 
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Table 4 - Estimated Effects of Minimum Competency Tests (MCT) and High Course Graduation 
Requirements (CGR) on Employment Participation and Wages by Race and Gender, 1990 PUMS 







Dependent Variable 






Employed 


Log Wages 


Sample 


MCT 


CGR 


MCT 


CGR 


All respondents 


-.0053* 

(.0031) 


.008 It 
(.0035) 


.0088 

(.0060) 


-.0074 

(.0070) 


White Males 


.0012 

(.0033) 


.0 1 06 J 
(.0041) 


.0094 

(.0069) 


-.0075 

(.0078) 


White Females 


-.0095f 

(.0031) 


.0011 

(.0044) 


-.0073 

(.0066) 


-.0041 

(.0070) 


Black Males 


.01641 

(.0069) 


.0339J 

(.0101) 


.0108 

(.0148) 


-.0279 

(.0177) 


Black Females 


-.0025 

(.0080) 


.0182* 

(.0104) 


.0169 

(.0133) 


-.0251 

(.0166) 



These estimates are based on linear probability models, which include fixed effects for race, gender, 
ethnicity, state of birth and year of birth as well as the controls for the unemployment rate, cohort size, 
poverty rate, real K-12 expenditures per-pupil and real post-secondary tuition. These models also include 
individual-level controls for educational attainment and student status. Heteroscedastic-consistent standard 
errors, adjusted for state-of-birth by year-of-birth clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
t Statistically significant at 5-percent level 
% Statistically significant at 1 -percent level 
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Table 5 - Estimated Effects of Minimum Competency Tests (MCT) and High Course Graduation 
Requirements (CGR) on Carnegie Units by Academic Subject, HS&B and NELS88 









Independent Variable 




Dependent 




1-2.99 Units 


3+ Units 


Academic Subject 


Mean 


MCT 


Required in Subject 


Required in Subject 


English 


3.7 


-.067 

(.056) 


.032 

(.051) 


-329J 

(.051) 


Social Studies 


3.1 


-.088 


-.015 


.133 


(.084) 


(.098) 


(.101) 


Science 


2.4 


-.135t 


.092* 


.393 J 


(.055) 


(.052) 


(.084) 


Mathematics 


2.7 


-. 1 27f 


-.020 


.110* 


(.050) 


(.046) 


(.059) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
f Statistically significant at 5-percent level 
l Statistically significant at 1 -percent level 
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Table 6 - Estimated Effects of Minimum Competency Tests (MCT) and Course Graduation Requirements 
(CGR) on Carnegie Units in Science and Mathematics 
by Race and Gender, HS&B and NELS88 



Sample 


Dependent 

Mean 




Independent Variable 


MCT 


1-2.99 Units 
Required in Subject 


3+ Units 

Required in Subject 








Science 




Full sample 


2.4 


-.135| 


.092* 


.3931 






(-055) 


(.052) 


(.084) 


Whites 


2.5 


-.075 


.109* 


.3381 






(.065) 


(.060) 


(.100) 


Blacks 


2.1 


-.225t 


.2411 


.5481 






(.093) 


(.113) 


(.084) 


Males 


2.5 


-.082 


.121* 


.4491 






(.060) 


(.067) 


(.108) 


Females 


2.4 


-.2091 


.064 


.3391 






(.057) 


(.053) 


(.072) 








Mathematics 




Full sample 


2.7 


-.1271 


-.020 


.110* 






(.050) 


(.046) 


(.084) 


Whites 


2.8 


-.062 


-.004 


.071 






(.053) 


(.046) 


(.064) 


Blacks 


2.5 


-.181* 


-.058 


.2511 






(.109) 


(.080) 


CH2) 


Males 


2.8 


-.128f 


.055 


.1731 






(.062) 


(.045) 


(.065) 


Females 


2.7 


-.157+ 


-.092 


.059 






(.060) 


(.069) 


(.073) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
f Statistically significant at 5-percent level 
X Statistically significant at 1 -percent level 
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Table 7 - Estimated Effects of Minimum Competency Tests (MCT) and Course Graduation Requirements 
(CGR) on Probability of At Least One Carnegie Unit by Academic Subject. HS&B and NELS88 



Independent Variable 





Dependent 




1-2.99 Units 


3+ Units 


Academic Subject 


Mean 


MCT 


Required in Subject 


Required in Subject 


English 


.979 


-.011* * 

(.006) 


.028J 

(.006) 


.029J 

(.006) 


Social Studies 


.964 


-.008 


-.002 


.003 


(.007) 


(.007) 


(.006) 


Science 


.951 


-.010 


.028 J 


.042} 


(.008) 


(.010) 


(.014) 


Mathematics 


.961 


-.001 


.003 


.009 


(.007) 


(.005) 


(.007) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
f Statistically significant at 5-percent level 
} Statistically significant at 1 -percent level 
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Table 8 - Estimated Effects of Minimum Competency Tests (MCT) and Course Graduation Requirements 
(CGR) on Probability of At Least One Carnegie Unit in Science and Mathematics 
by Race and Gender, HS&B and NELS88 



Sample 






Independent Variable 




Dependent 

Mean 


MCT 


1-2.99 Units 
Required in 
Subject 


3+ Units 
Required in 
Subject 








Science 




Full sample 


.951 


-.010 


-028J 


.042J 






(.008) 


(.010) 


(.014) 


Whites 


.956 


-.016* 


.029J 


.038 J 






(.009) 


(.010) 


(.016) 


Blacks 


.926 


-.008 


.043 1 


.068J 






(.014) 


(.018) 


(.012) 


Males 


.943 


-.001 


.029+ 


.035$ 






(.010) 


(.012) 


(.020) 


Females 


.958 


-.020t 


.029J 


.050J 






(.010) 


(.010) 


(.011) 








Mathematics 




Full sample 


.961 


-.001 


.003 


.009 






(.007) 


(.005) 


(.007) 


Whites 


.965 


.0003 


.002 


.006 






(.006) 


(.005) 


(.006) 


Blacks 


.937 


.016 


.022 


.021 






(.020) 


(.014) 


(.021) 


Males 


.954 


.003 


.008 


.0141* 






(.007) 


(.005) 


(.007) 


Females 


.967 


-.009 


.001 


.006 






(.009) 


(.009) 


(.011) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
t Statistically significant at 5-percent level 
{ Statistically significant at 1 -percent level 
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Table 9 - Estimated Effects of Minimum Competency Tests (MCT) and Course Graduation Requirements 
(CGR) on Carnegie Units in Calculus, Physics and Chemistry, HS&B and NELS88 









Independent Variable 






Dependent 




1-2.99 Units 


3+ Units 


Dependent Variable 


Mean 


MCT 


Required in Subject 


Required in Subject 


Calculus 


.08 


-,037t 


.003 


.023* 


(.015) 


(.009) 


(.013) 


Physics 


.20 


-.007 

(.019) 


.053+ 

(.014) 


.094{ 

(.017) 


Chemistry 


.46 


.026 

(.023) 


-.011 

(.022) 


-.007 

(.023) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post-secondary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 10-percent level 
t Statistically significant at 5-percent level 
X Statistically significant at 1 -percent level 
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Table 10 - Estimated Effects of Minimum Competency Tests (MCT) and High Course Graduation 
Requirements (CGR) on Arts and Use of Time, HS&B and NELS88 



Independent Variables 



Dependent Variable 


Dependent 

Mean 


MCT 


High CGR 


Carnegie Units in Visual and 


1.4 


-.071 


.022 


Performing Arts 


(.079) 


(.079) 


In Band, Orchestra or Chorus 


.28 


-.013 

(.012) 


.017 

(.012) 


Never or Rarely Reads For 


.37 


.023 


.041 + 


Pleasure 


C015) 


(.014) 


4 or More Hours of TV per 


.29 


-.007 


.046* 


Weekday 


(.014) 


(.009) 


Does No Homework 


.07 


. 011 * 

(.006) 


mot 

(.006) 



These regressions include fixed effects for state, year, race, gender, age, parental education, family 
composition, SES quartile as well as the state-year controls for the unemployment rate, cohort size, poverty 
rate, K-12 expenditures per-pupil, post- second ary tuition. Heteroscedastic-consistent standard errors, 
adjusted for state-year clustering, are reported in parentheses. 

* Statistically significant at 1 0-percent level 
f Statistically significant at 5-percent level 
t Statistically significant at 1 -percent level 
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